DOCOaSIT RESOMB 



ED 190 611 

AOTHOR 
TITLE 

POB DATE 
NOTE 



EDBS PRICE 
DESCHIPTOES 



IDENTIFIERS 



TH 300 403 

Boston, Margaret E« : Mitroff, Donna D. 
Continuing Effects: A Study of Knowledge 
Dtilization. 
Apr 80 

25p. : Paper presented at the Annual Meeting of the 
Aaerican Educational Research association (6^th, 
Boston, MA, April 7-11, 19B0). 

MP01/PC01 Plus Postage. 

♦Adoption (Ideay) ; Attitude Change; *Educational 
Innovation; Pollowup Studies; Inservice Teacher 
Education; Primary Education: *Program Effectiveness: 
♦Program iBpleoentation: Success; *Teacher Attitudes: 
♦Teaching Methods 

Eleaentary Secondary Education Act Title I 



ABSTRACT 

This Study examined the impact that the training in 
and the experience of using an innovative instructional program had 
on the practices and attitudes of teachers who were no longer in the 
supportive setting. Three years after two innovative Title l 
programs — Free Learning Program (based on the Bank Street ModelJ , and 
Primary Education Project— were terminated, teachers were asked to 
identify the program aspects, (instructional strategies, management 
techniques, curriculum components, etc,) which they continued to use 
in their current setting; Data were collected through mailed 
questionnaires and personal interviews: they indicated that program 
aspects persisted in the teachers • repertoire. Attributes of program 
aspects that contributed to program survival involved financial and 
social costs; efficiency: compatibility with conventional practices; 
complexity: amount of approval needed from other persons, or 
gatekeepers: and susceptibility to modification. Of the training 
which teachers received concerning the innovative programs, the 
all-dav inservice workshops were most enthusiastically supported. 
(Author/GDC) 
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The advent of the social reform movement of the sixties and federal 
intervention in education to help improve schools for the economically and 
educationally disadvantaged students generated an immense interest in all 
facets of the educational process. This movement resulted in an extensive 
effort in the design and development of educational products ranging from 
instructional curricula to new school organizational structures. Many such 
products were introduced to school districts as part of Federal programs 
that supported educational improvement, e.g., the Right to Read Program, the 
Follow Through Program. It was thought that the research based instructional 
innovations, combined with the massive Federal funding of the new programs 
would so impact the school classroom that dramatic changes in the face of 
education would soon result. 

Although innovative instructional programs permeate the educational 
scene today, little is known about what '.appens to the programs when they 
get into the schools or what impact they have on the people involved with 
them. These programs have frequently been evaluated by assessing and com- 
paring the academic progress of students through the use of pre and post 
measures. Test results are analyzed using standard statistical procedures 
and program effects are reported as the presence or absence of "significant 
differences". Although studies such as these can be informative, they don't 
provide an assessment of the total effect of the school change effort and 
frequently deny any program impact on the school environment. In addition 
to the measurement of the academic achievement of students, studies o^ pro- 
gram effects on teachers, educational practices, administrators, community, 
etc. need to be undertaken in order to gain an understanding of what happens 
to innovations in schools and what happens to the users. 

The research reported in this paper pertains to the broad school do- 
main, but focuses on one target for examining program outcomes, i.e., the 
ERIC o 



I 



2 



teacher; The overarching objective of educational research and development 
is to change classroom practices in order to improve instruction for the 
students. Nearly all instructional innovations have at least one common 
element: a teacher is required to implement the program. The general 
approach to this study is to regard the teacher as the actual consumer - 
the ultimate user of the innovation. It is the teacher who interacts daily 
with the student; it is the teacher who is the final "gatekeeper" through 
whom the innovation must pass. Therefore, the teacher should be an impor- 
tant area for the study of innovation impact. 

An additional dimension of this research is that long term or sus- 
tained program effects on teachers are examined. Although the major study 
• by Berman and McLaughlin (1978) looked at institutional and program factors 
that promote program continuation, follow-up studies of instructional in- 
novations have tended to focus on the school system as the unit of analysis 
and have concluded that if the innovation does not show as an item in the 
budget, the innovation is no longer present and change in the system has not 
occurred. However, one measure of the impact of any innovative instructional 
program should be the extent to which any of its components are so impres- 
sive that they have an extended life in the classroom despite school district 
termination of the program. 

The purpose of this research is to examine the long-term impact that 
the experience of being trained in and having used an innovative instructional 
program has on the practices and attitudes of the classroom teacher in the 
non-supportive setting. The term "Continuing Effects" is used to describe 
those features of an innovative program that are retained by the teacher, 
i.e., instructional strategies, management techniques, curriculum com- 
ponents, general principles, etc. Do any changes take place that are at- 
tributable to the program? Is anything still being used that was learned in 
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the program? The broader research effort addressed questions in four main 
areas: 

1) What program aspects are being continued or discontinued? 
What factors enhance or inhibit program continuation? 

2) What are the characteristics of the teachers who are con- 
tinuers-discontinuers? Is transfer related to teacher 
background characteristics e.g. educational level? 

3) What are the adaptations teachers make? What is the 
range of complexity? 

4) Is any second stage diffusion occurring i.e., from the 
teacher to other people in the current setting - what is 
the level of awareness, and attitudes of other school 
personnel about use of program aspects? 

Only the first area, "program aspect survival and factors that contribute 
to continuation", is dealt with in this paper. 

Setting 

This study is a systematic follow up of a group of teachers who were 
trained and taught in two innovative Title I programs before the programs 
were terminated by the school district. It is not an evaluation of program 
success or a comparison of the two programs. Rather, it is an attempt to 
describe how the process of program continuation works in the classroom 
setting and to identify some factors that contribute to long term usage of 
program features. This study was conducted in a large urban school district 
and is an extension of previous research reported by Mitroff and Boston (1977) 
as "Program Residuals - Or Did They Throw Out the Baby with the Bath." The 
residual study was conducted one year after program support was withdrawn; 
this current research was carried out during 1978>79 school year, three years 
after program termination. 

In 1959, as part of the ESEA Title I Program two different early 
childhood innovative instructional models were implemented in selected tar- 
get-area schools. One of the models, later known as the Free Learning Pro- 
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gram (FRELEA) , was developed locally by a group of school district personnel. 
It followed closely the Bank Street College open classroom model (Delano, 
1973). The other model was developed by the Learning Research and Develop- 
ment Center (LRDC) at the University of P-ttsburgh. This model, the Primary 
Education Project (PEP) combined a highly structured basic skills curriculum 
with more open-ended exploratory activities to foster personal and social 
growth (Wang, 1980'. PEP was In three schools while FRELEA was in twenty- 
line public and parochial schools. Both models or programs were introduced 
into one grade level each year until they were operating in grades K-3. 
During this time, the classroom teachers were receiving support for their 
efforts in the form of materials, training, and classroom aides. In 1976 
FRELEA and PEP were eliminated as budget items in the Title I contract. 
Method 

The approach to this study is descriptive field research through the 
use of the survey method. Data were collected by means of a mailed Ques- 
tionnaire and personal interviews conducted with a sample of former innovative 
program teachers. The Questionnaire was developed through an iterative pro- 
cess which included program developer/implementor consultations, use of per- 
vious findings, and pilot testing with program teachers. The items were de- 
signed to measure specific features of the programs. It was assumed that 
program features thus identified were not present in the teachers repertoire 
prior to the program, at least not to the extent that they were structured 
and organized in the program. The interview items were designed to examine 
major themes that emerged from the Questionnaire infonnation. 

Questionnaires were mailed to 130 teachers identified from the school 
rosters for 1975-76 school year, the program transition year, prior to term- 
ination. After a follow up mailing to non-responders, a total of 70 usable 
Questionnaires remained. 
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Twenty five interviews were conducted in ten schools by the sen^'or 
author. Anecdotal data supplied by the teachers in written form on the 
Questionnaire and verbally during interviews, were read and reread and 
classified many times. The data analysis was exploratory in nature. 
Since this study relied on quantitative and anecdotal data supplied by the 
teachers, the usual caution surrounding the limitations of self -reporting 
and retrospective reporting can be advanced. There is also a built in 
bias present in the data, since the responders are mainly program con- 
tinuers. At this time, there is no information abo„t the non-responders. 
Subjects 

The participants in the study were teachers who had had experience 
in either of the two innovative programs which had been terminated three 
years prior to the study. A profile of the "average participating teacher" 
was derived from the aggregate data collected by the Questionnaire on five 
background characteristics: 
Age: 35-36 

Total Years of Teaching Experience: 10-14 years 
Highest Degree Held: Bachelor's Plus 
Years at Current Grade Level: 6-10 years 
Years Teaching in Innovative Program: 4 years 

All but one of the participants in this Continuing Effects study remained 
within the school district setting where the innovative programs had been 
in operation. However, contextual conditions varied from teachers who were 
still teaching at the same greHe level in the same school with the same 
principal as when the program was in operation, to teachers where all these 
conditions had changed and to teachers who were in other than "classroom" 
situations. The majority of the teachers are still teaching in grades kin- 
dergarten through third grade. 
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RESULTS AND DISCUSSION 
The results of this study are organized around three major questions. 
First, what program aspects did the teachers use in the original operation 
of the program, and which of these aspects survive in the current setting? 
Second, are there any attributes of innovative program aspects that can 
enhance or inhibit their chances for survival? Third, are there any pro- 
gram characteristics that might account for the "continuing use" of know- 
ledge gained from the experiences of teaching in the innovation? 

Program Aspects 

The checklist on program aspects was organized around five major 
program Components which were common to both innovative programs; Physical 
Arrangements and Classroom Management. Curriculum, Record Keeping, Tech- 
nl.-ues for Individualizing, and Additional Resources. Within these five 
components, 30 PEP program aspects and 36 FRELEA aspects were originally 
identified and presented on the Questionnaire. Teachers were asked to 
check each aspect they had actually "used" in the original operation of 
the program. They were also asked if they "continue to use" the program 
aspect in their current setting. 

Use of Program Aspects in the Original Op eration of the Program. 

When the teachers were asked to identify specific program aspects 
they had used in operating the program, the majority of the teachers re- 
ported using a majority of the program aspects. A small number (5) of 
the aspects were reported utilized by so few teachers, that they didn't 
appear to be key aspects of the program. These aspects have been elimi- 
nated, since they could not provide information for further analysis of 
"continuing use". Five other aspects were regrouped for analysis. The 
remaining program aspects (56) are listed in Tables 1 and 2. Table 1 

o provides information about PEP program aspects and the responses of the 
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thirteen former PEP teachers. FRELEA program information is presented 
in Table 2. where a total of fifty>seven teachers responded. 

The number of teachers reporting "use" of an aspect in the original 
operation of the program is presented in the first column in both tables. 
It can be seen, for instance, by looking down the first column in the 
PEP Table 1, that eight program aspects were reported at the 100% level 
of usage. This means that thirteen out of the thirteen teachers said 
they had used the aspect when the innovative program was in operation. 
In the FRELEA Program, Table 2, six program aspects were reported used 
at the 100% level; fifty-seven of the fifty-seven teachers reported "use" 
in original operation of the program. 

A total of thirty-six aspects. (19 PEP and 17 FRELEA) were reported 
as "used" at the 90% level or above in the original operation of the program. 
Reported usage of aspects in Tables 1 and ? ^^ringed from 100% usage for 
about one-forth (14) of the individual aspects to a low of 61% for one PEP 
aspect (Selected/Directed/Skills). Variations in levels of usage of specific 
program aspects has also been found in studies measuring the degree of im- 
plementation during actual program operation (Fullan and Pomfret. 1977). In 
spite of the variation in usage, this group of teachers identify these pro- 
gram aspects as having been used during the actual operation of the program. 
Continuing Use of Program Aspects 

On the Questionnaire teachers were asked if they continue using aspects 
in their current school setting. The number of teachers reporting "con- 
tinuing use" is presented in Co1u"^ n two of Tables 1 and 2. 

As an aid in handling the data, an Index of Continuation was develop- 
ed for each program aspect. It was calculated as a ratio of those reporting 
Continuing Use to those reporting Use in Original Operation of Program. The 
Index of Continuation (IOC) is displayed in column three in Tables 1 and 2. 
ERIC 9 
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Number of PEP Teachers Reporting Initial Use and Continuing 
Use of Specific Program Aspects (N«13) 



Number of Teachers Reporting 
Use ^n Original Index of 

Operation of Continuing Continuation 

Program Aspects ProBTSIL. ^-^ ' ^ ' 

Physical Arrangement and 
Classroom Managemenf 

Tj* 12 92 
Exploratory area *5q 

Prescriptive area 12 o 

Testing center ' ,*qq 

Positive Reinforcement Jf ^*^« 

"Flags" ^ 12 5 .^^ 

Systematic Behavior Management U ^ 

System for assigning areas 12 ^ 

Learning Centers 12 

Curriculum 

IPI Reading 12 3 .25 

IPI Math . ]l 7 70 

Classification/Quantification 10 ' ^-^q 

Perceptual Skills 9 ^ iss 

Sound/Blend System i^ 50 

0 8 
12 6 



Selected/Oirected/Skills B ^ -g^ 

Math Mainteance i" . *50 



Manuals 
Record Keeping 



Additional Res ources 
Classroom Aide 



9 .75 



Placement ]l 3 .56 

Cumulative record |^ 5 . 50 

Weekly assignment sheets lu ^5 

Progress profiles l^ 

I ndividualizing Techniques 

if 13 1-00 

Pre-tests ^2 .92 

Post-tests ^2 .92 

Placement Tests i-^ 

Curriculum Embedded Tests-CETS 10 J 

Prescription (tickets) 11 ^ -^2 

Traveling 13 ^ '^O 

Cassette Instruction 10 ^ 



12 8 .66 



* Program aspects displaying 100% usage in the original operation of the program, 
ERIC ^ " 
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Number of FRELEA Teachers Reporting Initial 
Use and Continuing Use of Specific Program Aspects (N=57) 



Program Aspects 



Number of Teachers Reporting 

Use in Original 
Operation of Continuing 
Program Use 



Index of 
Continuation 
Col . 2 Col . 1 



Physical Arrangement and 
Classroom Manaqment 

Desk or tables grouped 
Children's Work displayed 
Flexibility of time, space 
Small Group Instruction 
Independent Activities 
Peer Teaching 
Projects 

Group Charts, Checklists 
Learning Centers 

Curriculum 

Teacher Made Materials 
Music Activities 
Art Activities 
Eurythmi cs 
Creative Writing 
Language Experience 
Math Activities 
Perceptual Activities 

Record Keeping 

Teacher/Pupil Folders 
Anecdotal Records 
Teacher/Pupil Files 
Charts, checklists 

Individualizing Techniques 

Diagnostic Use of MAT/Ginn 
Teacher made CR Tests 
Observation 
Activity Cards 
Independent Study 
Cassette Instruction 

Additional Resources 

Classroom Aide 



56 

57* 

56 

57* 

57* 

49 

44 

53 

54 



57* 

49 

56 

46 

47 

56 

57* 

53 



57* 
40 
43 
55 



40 
45 
56 
53 
41 
46 



53 



49 
56 
54 
55 
56 
49 
37 
48 
53 



53 
39 
52 
24 
45 
54 
54 
52 



57 
37 
39 
54 



40 
45 
SZ 
46 
35 
35 



31 



.88 
.98 
.96 
.96 
.98 
1.00 
.84 
.91 
.98 



.93 
.80 
.93 
.52 
.96 
.95 
.95 
.98 



1.00 
.92 
.91 
.98 



1.00 
1.00 
.93 
.87 
.85 
.76 



58 



* Program aspects displaying 100% usage in the original operation of the program. 
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A review of the IOC indicates that the degree of continuation differs by 
aspect. The extent of aspect continuation ranges from .25 to 1.00, an 
IOC shared by seven different aspects. For the most part, though teach- 
ers report a substantial continuation of innovative program aspects; only 
five aspects show less than 50% continuation. How can this situation be 
explained in light of the elapsed time since program termination and the 
absence of resources and support mechanisms? Why are teachers able to 
continue aspects? What are the factors that enhance or inhibit con- 
tinuation? 

Aspect Attributes 

Just as the studies on the degree of program implementation suggest 
that some innovative program features are implemented more often than 
others, the data from this study indicate that some program aspects are 
continued -nore often than others. 

In this section individual program aspects will be examined to 
see if those aspects which show either a high IOC or a low IOC have any 
attributes in common. While much of the research has focused on the 
early "adoption" stage of the innovation (Carlson, 1965; Miles, 1954; 
Rogers and Shoemaker, 1971), few major studies have explored the latter 
"continuation" stage. For this reason the literature on the adoption 
phase was examined for innovation attributes which can be applied to the 
data on program aspects. 

Zaltman, Duncan, and Holbek (1973) discuss attributes of innova- 
tions which "have been found to be relevant for describing, explaining, 
and predicting responses to innovations." (p 33) Their discussion covers 
the potential impact of nineteen attributes on the adoption of an innova- 
tion. These attributes were originally posed by Lin and Zaltman (1973). 

ERIC X Z 



While all nineteen innovation attributes have impact on the adop- 
tion phase, they do not all have impact on the continuation phase. There 
are seven attributes which, in our judgment have an impact on the con- 
tinuation phase; they are: financial costs, social costs, efficiency, 
compatability, complexity, gatekeeper involvement, and susceptibility t-» 
modification. 



1. Financial cost is one of the most obvious dimensions that would 
impact the continuation stage. Would the user have to assume 
costs in order to transfer this particular program aspect? 

Social cost, being another form of expense, may also have an 
impact on the user's decision to continue or discontinue a 
particular urogram aspect. Under social costs are included such 
things as being subject to ostracism or ridicule by colleagues. 
Would the user be recognized or set apar\. as odd or different 
if this aspect were present in the classroom? 

3. Efficiency refers to "overall time-saving and avoidance of 

bottle-necks". Would the aspect contribute to the users comfort- 
ableness and the efficient management of time? 

Compatibility , according to Lin and Zaltman, concerns the sirailar- 
ity of the innovation to the existing product which it may 
ultimately supplement, complement, or replace. In the case of our 
study it refers to the compatibility of these program aspects with 
traditional instruction (TI). Can the program aspect be interspersed 
with traditional instructional models without conflict? 

5. Complexity is dealt with on two levels. Does the program 
aspect embody complex ideas? And/or is it complex in its 
actual implementation? 

6. Gatekeeper involvement refers to the number of "gatekeepers" 
e.g. principals, supervisors, etc. who need to be passed or 
"trespassed" before the user can employ the practice/pro- 
cedure in the current setting. Does the use of the aspect 
require active involvement of others outside the classroom? 

7. Susceptibility to modification concerns whether or not the 
program aspect can be modified or adapted to current settings. 
Can it be integrated with the curriculum which is in use in 
the current setting? 
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The program aspect/attribute analysis procedure described here is 
an exploratory one. Research on this topic is still in the early stages, 
so there is a need to explore methodologies that can be refined by future 
research. 

As an outcome of discussions with colleagues concerning the seven 

attributes, and based on the authors' collective knowledge and experience 

in dealing with all phases of an instructional innovation's life cycle, an 

Ideal Pattern of Attributes was formulated. The question asked was whether 

the presence (+) or absense (0) of the attribute would contribute to the 

chances of survival of the program aspect. The pattern of positive/negative 

attribute designations which would yield the best chances for continuation 

of program aspects is presented in the following list. 

Financial Costs 0 

Social Costs 0 

Efficiency * 

Compatibility ^ 

Complexity 0 

Gatekeeoer Involvement 0 

Susceptibility to Modification + 

The attributes of Financial Costs, Social Costs, Complexity, and Gatekeeper 
Involvement have all been designated as zero (0), since the presence of 
these attributes in an aspect represent factors that could inhibit contin- 
uation. On the other hand. Efficiency, Compatibility, and Susceptibility 
to Modification have been designated as plus (+), since they represent 
factors that could enhance continuation. 

The next step in the examination of program aspects and innovative 
attributes was to select eight aspects showing a high IOC and eight show- 
ing a low IOC. Included in the final list were aspects from both programs 
and from each component. A matrix was set up, and the authors proceeded 
independently 1n the completion of the matrix. In each cell an assignment 
of a plus or zero was made dependent upon the judgment as to whether the 



program aspect possessed the attribute. After completing the matrix in- 
dependent!: , each cell was discussed and consensus reached on those items 
where there was a difference of opinion. The results of this procedure 
are displayed in Table 3 where the eight high and the eight low continuing 
program aspects are presented in light of the seven innovation attributes. 

Table 3 was then examined to see if any different characteristics 
emerge for high continuing aspects and low continuing aspects. First* each 
column was inspected for similarity and differences in a ttributes of the 
two groups of aspects. Second, the two groups were compared to the Ideal 
Pattern of Attributes and some generalities were drawn. 

A ttributes . High Continuing Aspects (HCA) display a more consistent 

designation of attributes than the Low Continuing Aspects (LCA). 

Financial Costs . Financial Costs do not have to be assumed by the 
user when using HCA. A financial cost is associated with two LCA. 

Social Costs . Only one HCA, Exploratory Area, entails any social 
costs; three LCA exhibit this attribute. 

Efficiency . All HCA contribute to the efficient use of time; only 
three LCA do. 

Compatibility . All HCA except one. Exploratory Area, are seen as 
compatible with traditional instruction; only two LCA are. 

Coipplexity . HCA show a mixture of positive/negative designations; 
all LCA are seen as complex. 

Gatekeeper Involvement . Only one HCA and one LCA require gatekeeper 
involvement. 

Susceptibility of Modification . All HCA are susceptible to modification 
only three LCA display this attri bute . 

Program Aspects . The group of high continuing aspects possess at- 
tribute designations that are closer to the Ideal Pattern of Attributes than 
the low continuing ones. In fact, five HCA display the ideal pattern of 
positive and negative attribute designations; none of the low continuing 
aspects show this pattern. One low aspect, IPI Reading, shows a totally 
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Table 3 



Innovative Attributes Impacting Continuing Program Aspects 



Program Aspects 



Hicyh Continuing Aspects (HCA) 
Positive Reinforcement* 
Pre-tests 
Peer teaching* 
Teacher/Pupil Folders* 
Diagnostic Use Mat/6inn 
Exploratory Area 
Perceptual Activities* 
Creative Writing* 



Low Continuing Aspects (LCA) 
IPI Reading 
Cassette Instruction 
CET's 

Eurythmi cs 
Prescriptive Area 
Weekly Assignment Sheets 
Music Activities 
Testing Center 



Ideal Pattern 



Attributes 



Financial 
Costs 


Social 
Costs 


Efficiency 


Compatibll ity 


Compl exi ty 


Gatekeeper 
Involvement 


Susceptibility 
to Modification 


0 


0 




+ 


0 


0 


+ 


0 


0 


+ 


+ 


+ 


0 


+ 


0 


0 


+ 


+ 


0 


0 


+ 


0 


0 


+ 


+ 


a 


0 


+ 


0 


0 


+ 


+ 


+ 


+ 


+ 


0 


+ 


■I- 


0 




0 


+ 


0 


0 


+ 


+ 


c 


0 


+ 


0 


0 


+ 


+ 


0 


0 


+ 


+ 


+ 


0 


0 


+ 




0 


+ 




+ 


0 


+ 


0 


0 


0 


0 


0 


0 


+ 


0 


0 


0 


+ 


0 


0 


+ 


0 


+ 


0 


0 


+ 


0 


+ 


0 


0 


0 


0 


+ 


+ 


+ 


0 


+ 


0 


0 


0 


+ 


+ 


0 


+ 


0 


0 


0 


0 


+ 


0 


0 


0 


0 


1 


+ 


0 


0 


+ 



Note: A plus (+) means the attribute is present in the program aspect, 

A zero (0) means the attribute is not present in the program aspect, 

*Aspects that natch the Ideal Pattern 
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opposite pattern from the ideal. 

The preceding discussion shows that it is possible to distinguish 
among High Continuing Aspects and Low Continuing Aspects. Program Aspect 
continuation is not dependent on any one attribute for survival. However, 
an ideal pattern of attributes, such as the one presented here, could pro- 
vide some guidelines for predicting program aspect survival. Such infor- 
mation has implications for program developers. The closer a program aspect 
resembles the ideal pattern of positive/negative designations of innovation 
attributes, the better its chance for survival in the teaching practices of 
the classroom teacher. There is no guarantee, though, that if an aspect 
displays the ideal pattern, or near ideal pattern, that it will be continued. 
Aspect attributes are just one factor that can contribute to the survival 
or discontinuation of innovative programs. 

Program Characteristics 

The two innovative programs examined in this paper were products of 
a systematic implementation plan that included training, monitoring of pro- 
gress, special materials, a classroom aide, and other professional resources. 
In this section some comments made by teachers about the program resources 
will be presented and resources will be discussed in terms of their con- 
tribution to "continuing use" of the program. 

During the time the two programs were in operation, five resource 
and su pport mechanisms were provided in the form of professional training 
and personnel: 

Sun wer Workshops . These Program specific workshops were con- 
ducted prior to school opening and were separate from the 
regular two day pre-service days. 

All day in -service workshops . These regularly scheduled work- 

shops were held during the school year; they were f^^^^ted 

by Program staff who were responsible for program implementation. 

ERIC ^ « 
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Team Leader . This person functioned as the Program on-site 
facilitator within one school. 

Superv isory Instructional Specialist . The person ^as the 
administrator appointed "Supervisor" who was responsible 
for all teachers in 5 to 6 schools. 

Program Staff . People In this category were specially 
trained in the Program and were responsible for teacher 
training and Program implementation. PEP field repre- 
sentatives were "outside" consultants. FRELEA staff ware 
School Board employees. 

Teachers were asked to rate these five program resources on a scale 
of 0 to 5 in terms of their value as resources for professional growth 
during the program experience. All resources except the Supervisory In- 
structional Specialist had mean ratings of 3.0 or above. "All Day In- 
Service Workshops" consistently maintained the highest rating. These work- 
shops were also one of the "most missed" program aspects reported by the 
teachers in the interviews. Teachers said they valued these occassions be- 
cause they provided the opportunities to exchange ideas and techniques; pro- 
vided access to games, materials, and equipment; and were a source of stim- 
ulation, incentive, and motivation. 

I really miss the all day in-service workshops . 
They were a constant incentive to try new ways 
of teaching, using new materials. They also kept 
me in touch with what was going on in the public 
schools. 

The Program staff, FRELEA workshop staff and/or the PEP consultant 

staff, was rated next highest. These groups of people provided in-service 

and program support. Teachers commented that they missed the professional 

interaction with these people, which constituted support for the program. 

I really miss the Program. I have always valued the 
workshops and I was also impressed with the people who 
were respon sible for the workshops each month. I felt 
they put a lot of thought and work into making thern 
interesting and instructional. The program changed my 
attitudes to teaching in the classroom very much. 
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Summer workshops received the third highest rating. The Team Leader 
(fourth) and the Supervisory Instructional Specialists (fifth) were close 
in their ratings. The Supervisory Instructional Specialist was not a pro- 
gram specific function. The Team Leader, however, was unique to the program. 
Teacher comments suggest that the usefulness of the Team Leader function 
was related to the person performing it: 

I also miss having a Team Leader to consult with. 
Sometimes it is necessary to have a sounding board 
for ideas — and it works out even better if that 
person is totally familiar with the classroom. 

Three different Teem Leaders with three different 
philosophies were in charge over the five-year 
period. Each one emphasized a certain area: Learn- 
ing centers, activity cards, independent activities, 
contracts, etc., so I've dealt with almost every item 
mentioned here in the checklist. My reactions each 
year depended a lot on the input I was allowed (or not 
allowed) and amount or lack of pressure to fulfill 
some of my own needs as a creative person. 

It is interesting to note that rankings of program resources by these 

teachers are quite similar to the rankings of the very same items in the 1976 

Residual Study (Mitroff and Boston, 1979). The passing of time doesn't seem 

to have diminished the teachers' perception of program experiences in regard 

to the support structures. 

Both of these innovative programs required the teachers to make their 
own materials in order to implement the program. Not only do many of these 
materials survive termination, but teachers report they continue to create 

their own materials. 

I still have thousands of task cards I made in the 
Program - I'll never give them up! In the Program 
v/e had a lot of workshops , materials , and had time 
to make good materials. 

I spent hours a nd weeks l ast summer making my 
own materials — did it all or my own time. 
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We put all of the auditory motor activites on 
individual cards - spent the whole summer typing 
every single thing from the auditory book. 

Activities such as making materials during the program operation, can 
contribute to program continuation in many ways. 

As a result of the experiences and training in the program, teachers 
built up attitudes that can influence their use of the program. The general 
theme of positive committment repeatedly emerged in both the interviews and 
comments supplied by teachers on the Questionnaires. Even those teachers 
who commented negatively about particular program aspects and the program ir 
general reported continuing use of techniques learned in the program. Here 
are some statements teachers made about the program: 

The program was very beneficial to me as a teacher. 
The techniques and ideas used in FRELEA helped me 
grow as a teacher. found that with the help of 
FRELEA activities, workshops, and team leaders, I 
was able to benefit myself as well as the children. 
Just the sharing among different teachers was very 
beneficial . 

As is obvious from my responses, I believed in the 
PEP Program and still do . I would love to bring it 
back completely, but I do manage to keep as much of 
it going as I can. 

Teacher committment to the program can also be illustrated by their 

predictions of how long they will continue to use the program. Invariably. 

when teachers were asked the question about how long they could continue, 

they replied that they would always use it . Here are some teacher replies: 

I'll continue to use program aspects indefinitely , you 
have to adapt, but I'll still keep using it. For exam- 
ple, in decoding, I may not use all the steps, but I 
use it; it's internalized . 

I won't stop using the program aspects because I've 
become comfortable with it and the kids are now self- 
relient, they're not constantly waiting for the next 
direction. I'll change only if a better program comes 
along. 
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It appears that one of the most powerful determinents of innovative 
program continuation can be teacher cornmittment to the philosophy, goals, 
and operation of the program. This committment is brought about by the 
experiences encountered during the life of the program, and can enhance 

survival of the program. 

In the Rand Report (1978) Berman and McLaughlin advance seven elements 
of locally chosen implementation strategies that enhance program implement- 
ation and continued use of the program at the classroom level (p. 34). The 
data from the Continuing Effects study were reviewed in order to determine 
if any factors similar to these seven elements emerge as contributors to the 
continuance of program aspects. The elements and discussion follow. 
Element 1: Concrete, teacher specific, and extended training. 

The innovative programs examined in this study give evidence 
of having provided program/ teacher specific in-service. Training 
was provided throughout the entire time the programs were in operation. 
Element 2: Classroom assistance from project or district staff . 

Support systems were integral parts of the programs, and the 
data suggest that they played a significant role in the program 
operation. This element combined with element one, probably ac- 
counts for teacher commitment to the programs. 

Element 3: Teacher observation of similar proj ects in other 
classrooms, schools, or district^ - 

Although information on this element was not specifically re- 
quested by the study, it is known that observation, demonstrations, 
and sharing workshops were part of the teacher training. There are 
teacher statements in the data to substantiate this element. 
Element 4: Regular pro,^ect meetings that focused on pract ical problems. 

Once again this element is not necessarily addressed in the 
study. But in addition to the "All day in-service project workshops". 
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school project meetings were conducted regularly by the team leader. 
Element 5: Teacher participation in project decisions . 

This study did not address the question of teacher participation 
in project decision making. However, what it does show is that teach- 
ers frequently question the decision making process, but adjust to 
the consequences. 

Element 6: Local materials development . 

One very important feature that emerges from the data is that 
the Program teachers were heavily involved in materials development; 
the time, the supplies, and the training for such an effort were 
all readily available. 

Element 7: Principal participation in training . 

Although the data do not reveal this element directly, it is 
known that during the program operation, both orientation and train- 
ing were available for the principals. 

As can be seen from the preceding discussion, the data from the present 
study support all but one cf the seven elements presented by Berman and Mc- 
Laughlin. One additional element should be added to the list if it is to be 
used as a framework for examining continuing effects. That element pertains 
to the Time the program was in operation and the Time the teacher taught in 
the innovative setting. The two programs in this study lasted a number of 
years - they were not short term innovations. Teachers need time to "inter- 
nalize" the new experience. Hall and Loucks, (1977) Loucks (1978) propose 
that it takes teachers varying amounts of time to reach the different stages 
in their eight Levels of Use. They further suggest that it takes one and a 
half to two years of teaching in a new program before teachers move beyond 
the Mechanical Use stage. Innovative programs should be given a reasonable 
amount of time to be adopted, implemented, and operational i zed before any 
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program impact or continuing effects can be assessed. 

Summary 

In this paper » program aspects from two innovative programs were ex- 
amined to see if any survive in the teacher's practices in the classroom. 
An Index on Continuation (IOC) was developed as a means of identifying the 
survival rate of aspects in the current setting. It was seen that the teach- 
ers in this study report continuing a high number of program aspects even 
after three years have elapsed since the school district's official termination 
of the program. Given that teachers report continuing use of aspects, at- 
tributes of the aspects themselves and characteristics of the programs were 
examined to see how they contribute tj continuation. 

An Ideal Pattern of Aspect Attributes was formulated based on seven 
innovation attributes proposed by Lin and Zaltman (1973). A group of high 
and low continuing aspects were compared to each other and to the Ideal 
Pattern. It was brought out through this procedure, that attributes of in- 
novative program aspects can help determine the chances for survival. 

Seven elements that enhance implementation and continuation (Berman 
and McLaughlin, 1978) were compared to known program characteristics. All 
but one of the elements was accounted for by the program characteristics. 
It has been suggested here that coimiittment to the principles of the "program" 
accumulated through training, on-site assistance, materials making, and ex- 
perience using the program played an important part in the continuation re- 
ported by the teachers. 

Although the data in this paper may be criticized by the self -reporting 
nature and the lack of measurement of the frequency of the continuing use 
of the aspects, data gathering techniques used in this study are sufficient 
to provide some insights into what happens to innovations and the users. 
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The paper serves to illustrate that knowledge gained in the innovative pro- 
grams is still utilized by teachers. 

CONCLUSIONS 

In this paper it has been argued that innovative program outcomes 
should be determined by examining effects on the total school environment 
in addition to the usual student achievement measurement. This stance was 
taken in the belief that such documentation constitutes a legitimate con- 
tribution to the determination of knowledge utilization and a refutation to 
the claim that nothing can be changed by R & D. Since there is no real theory 
or clear definition of knowledge utilization we have adopted an operational 
definition which asks the question "How do you know that knowledge is being 
used?" And we answer that, if after three years since program termination, 
teachers who particpated in the innovative environment can still relate to, 
identify, and articulate how their current practices were affected by such 
participation, then knowledge is being used. The data from this rather unique 
longitudinal study demonstrates that teachers are able to report what is being 
utilized. We rest our case on the empirical data: what the teacher, the 
ultimate user of the instructional innovation, says is happening. As one 
teacher so profoundly commented: You don't think we're going to throw away 
everything we learned! Do you? 
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